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TIMELINE-BASED GRAPHICAL USER INTERFACE FOR EFFICIENT 
IMAGE DATABASE BROWSING AND RETRIEVAL 

FIELD OF THE INVENTION 

5 The present invention relates to graphical user interfaces in general, 

and in particular to graphical user interfaces adapted for browsing and retrieval of 
pictures in digital picture databases. 

BACKGROUND OF THE INVENTION 

10 Digital pictures and digital picture albums/collections are prolifer- 

ating with recent advances in quality and cost reductions. Digital pictures can be 
stored in common or individual databases that allow a user to retrieve his or her 
pictures for viewing and reproduction via a computer system. When a large num- 
ber of pictures are stored in a collection, it is desirable to be able to browse and 

15 retrieve selected pictures with a graphical user interface (GUI). Used for such an 
application, a GUI conveniently organizes and groups database pictures, and al- 
lows a user to browse the pictures in one or more displays. 

Along with the digital information representing pictures, digital 
image files can also contain metadata, or information not displayed, which none- 

20 theless characterizes the data comprising the pictures. The metadata can describe 
the scene in the picture, such as a caption, and can also provide in a straightfor- 
ward manner, information such as the date and time the picture was captured, the 
location from which the picture was captured, identify people or objects in the 
picture, and information regarding format and data structure. 

25 Many prior art digital cameras can be programmed to automatically 

store along with the actual image, the date and time a picture was captured. More 
advanced digital cameras can also be programmed to automatically store along 
with the actual image, the location of picture capture by harnessing automatic lo- 
cation systems. For example, the Global Positioning Satellite (GPS) is a well- 

30 known method for pinpointing the location of a special GPS receiver with a fairly 
high degree of accuracy. Other methods include the use of Radio Triangulation 
(RT) systems. Using such an approach, a GPS receiver can be either incorporated 
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in the hardware of the digital camera, or located nearby. A subsequent image file 
will contain not only the raw image data, but also a date and time stamp, along 
with header information related to the location of the GPS receiver when the im- 
age is collected. 

5 Still more advanced digital cameras may contain pattern recogni- 

tion software for identifying objects and people in an image, and converting such 
information to metadata. 

Whether metadata is entered automatically by the camera, manu- 
ally via the camera during picture captures, or subsequently into the digital image 
10 database, there remains a great need for improved GUIs for efficiently and attrac- 
tively arranging pictures in a digital image database. More particularly, there re- 
mains a need for improved, user- friendly GUIs that give image database users a 
temporal "feel" for the stored images. 

1 5 SUMMARY OF THE INVENTION 

To address the needs of the prior art identified supra, the present 
invention provides a graphical user interface adapted to browse and retrieve pic- 
tures stored in a digital image database. The graphical user interface includes a 
metadata decoder adapted to decode metadata stored in digital image files, a first 

20 display level and at least a second display level, which display levels are con- 
structed with the aid of decoded metadata. The first display level at least includes 
a first timeline metaphor representing the span of time over which pictures stored 
in the database were captured, and a plurality of first icons placed along the first 
timeline metaphor representing predefined temporal intervals, the first icons being 

25 proportionately sized to correspond to the number of pictures captured during a 
predefined temporal interval. The second display level is linked to the first dis- 
play level and is triggered by activating one of the first icons. The second display 
level at least includes a second timeline metaphor, and second icons placed along 
the second timeline for grouping the pictures represented by the activated first 

30 icon, the second icons being proportionately sized to correspond to the number of 
pictures captured for each grouping. 
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The present invention also provides a method of producing a 
graphical user interface (GUI) adapted to browse and retrieve pictures stored in a 
digital image database. The method includes the steps of decoding metadata 
stored in digital image files, providing a first GUI level and at least a second GUI 
5 level, which GUI levels are constructed with the aid of decoded metadata. The 
first GUI level at least includes a first timeline metaphor representing the span of 
time over which pictures stored in the database were captured, and a plurality of 
first icons placed along the first timeline metaphor representing predefined tempo- 
ral intervals, the first icons being proportionately sized to correspond to the num- 

10 ber of pictures captured during a predefined temporal interval. The second GUI 
level is linked to the first GUI level and is triggered by activating one of the first 
icons. The second GUI level at least includes a second timeline metaphor, and 
second icons placed along the second timeline grouping the pictures represented 
by the activated first icon, the second icons being proportionately sized to corre- 

15 spond to the number of pictures captured for each grouping. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Features and advantages of the present invention will become ap- 
parent to those skilled in the art from the description below, with reference to the 
20 following drawing figures, in which: 

Figure 1 A is a yearly-based timeline display used in the graphical 
user interface (GUI) of the present invention, the timeline having chronological 
icons which are proportional in size to the number of pictures in a database corre- 
sponding to the identified years; 
25 Figure IB is an alternate decade-based timeline display suitable for 

use in the GUI of the present invention, the timeline having chronological icons 
which are proportional in size to the number of pictures in a database corre- 
sponding to the identified decades; 

Figure 2A is a monthly-based display linked to the timeline in Fig- 
30 ure 1 A in general, and in particular to a year specified by a user; 

Figure 2B is a yearly-based timeline display linked to the timeline 
in Figure IB, which has chronological icons which are proportional to the number 
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of pictures in a database corresponding to the years in a decade specified by a 
user; 

Figure 3 is a weekly-based (Sunday through Saturday, etc.) display 
linked to the timeline in Figure 2A in general, and in particular to a month speci- 
5 fied by a user; 

Figure 4 is a daily-based (by hour of the day) display linked to the 
timeline in Figure 3 in general, and in particular to a day of the week specified by 
a user; 

Figure 5 is an alternate daily-based (by blocks of time) display 
10 linked to the timeline in Figure 3 in general, and in particular to a day of the week 
specified by a user; 

Figure 6 is an event-based display linked to the timeline in Figure 
2A in general, and in particular to a month specified by a user; 

Figure 7 is an alternate embodiment (vertical) of the display in 

15 Figure 6; 

Figure 8 is a location-based display linked to the timeline in Figure 
2A in general, and in particular to a year specified by a user; 

Figure 9 is a person-based display linked to the timeline in Figure 
2A in general, and in particular to a year specified by a user; 
20 Figure 10 is a thumbnail display of pictures in a database that are 

linked to the timeline displays of the previous drawing figures; and 

Figure 1 1 is a conceptual block diagram of a system used for pro- 
ducing a GUI according to the present invention. 




25 DETAILED DESCRIPTION OF THE INVENTION 

As will be described below in more detail, the present invention 
novelly provides a graphical user interface (GUI) which uses a series of linked 
timeline metaphors with proportionately (with respect to the number of pictures 
represented by the icon, as will be explained, infra) sized icons to represent the 
30 pictures in a digital image database. It will be appreciated by those skilled in the 
art that the pictures in the image database can be entered by a variety of means, 
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including, inter alia, pictures captured by digital cameras, analog pictures which 
are quantized into digital images, and images produced by image scanners. 

The first level of the GUI of the present invention contains a time- 
line metaphor (displayed for the user) such as the one 100A in Figure 1A 5 or the 
5 one 100B in Figure IB. The timeline metaphor 100A is marked by the years (of 
capture) covered by the pictures of the database in question. In an alternate em- 
bodiment, the timeline metaphor 100B is marked by the decades (of capture) cov- 
ered by the database pictures. More particularly, the timeline 102 A has year- 
picture icons 104 A evenly spaced with year markers, while the timeline 102B has 

10 decade-picture icons 106B evenly spaced with decade markers. 

Although shown as circular in shape, the icons 102A can be of any 
other suitable shape, including (but not limited to) spherical, elliptical, or irregu- 
lar. More important to the present invention than the shape of the icons 104A and 
106B is the fact that the icons are sized to show the relative number of pictures 

15 represented by the corresponding time markers. For example, the sample digital 
image database contains many more pictures captured in the year 2001 than in the 
year 2000. Consequently, the icon representing to the year 2001 is relatively 
large, while the icon representing the year 2000 is relatively small. In the alter- 
nate embodiment of Figure IB, the number of pictures in the sample database 

20 captured in the 1991-2000 decade are relatively large, while the number of pic- 
tures captured in the 1941-1950 are relatively small. Consequently, the icon sizes 
are relatively large, and relatively small, respectively. 

To move to the next level of the GUI of the present invention, a 
user activates one of the year (or decade) icons in Figures 1 A or IB to produce a 

25 new timeline metaphor 200A or 200B shown in either Figures 2A or 2B for 

greater temporal specificity. Just as with Figures 1 A and IB, the timeline meta- 
phors 200 A and 200B contain timelines and relatively sized picture icons. 

In the example shown in Figure 2 A, the user has activated (by a 
click of a pointing device such as a "mouse," for example) the year 2000 icon as 

30 being of further interest. That is, the user is interested in browsing pictures cap- 
tured in the year 2000. The new timeline 202A further refines the year 2000 into 
the months of the year. From the example, it is apparent by the relative size of the 
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icons 204 A, that many more pictures were captured in the month of December 
than in the Month of May. In the alternate timeline metaphor 200B of Figure 2B, 
the user has chosen to further refine the timeline metaphor of Figure IB, by 
showing the year-picture icons 208B on the timeline 206B to display the relative 
5 size of pictures captured for each of the years in the 1991-2000 decade. In this 
example, the pictures captured in the year 1 999 are much more numerous than the 
pictures captured in the year 2000, so that the relative sizes of the corresponding 
icons are large and small, respectively. 

Activating an icon representing the picture grouping of interest can 

10 further refine the timeline metaphors of each level of the present-inventive GUI. 
The timeline metaphor 300 in Figure 3 is linked to either the timeline metaphor 
200A or 200B, and groups the pictures in the image database according to the 
days of the week with a timeline 302 and day icons 304, which are, again, propor- 
tionately sized to represent the relative number of pictures grouped according to 

15 the days of the week. The timeline metaphor 400 in Figure 4 is linked to the 

timeline metaphor 300 in Figure 3, and groups the pictures in the image database 
according to the time of day with a timeline 402 and day icons 404. In the exam- 
ple shown, the pictures captured on all Mondays are grouped according to their 
capture time. An alternate embodiment of the timeline metaphor 400 in Figure 4 

20 is the timeline metaphor 500 in Figure 5, where instead of grouping the pictures of 
a particular day by the hour of capture, the pictures are grouped by a block of 
time. 

The GUI of the present invention allows a user to display meta- 
phors which group the pictures in a digital image database in other ways besides 

25 the date or time of capture by linking timeline metaphors to other metaphors. The 
display line 602 in Figure 6 displays proportionately sized icons 604 which group 
the database pictures according to events specified by a user inputting the pictures. 
Linked to the timeline metaphor 200A in Figure 2A, the metaphor 600 in Figure 6 
groups the pictures captured in a particular month— December in the example — 

30 into identifiable events, such as birthdays, holidays, etc. For example, the "Event 
11" icon might represent pictures captured on Christmas Day, while the "Event 
12" icon might represent pictures captured on New Year's Eve. 
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While the metaphors of the GUI of the present invention might 
generally be oriented in a common direction, different levels might be displayed 
in an orthogonal direction relative to previous linked levels, such as the metaphor 
700 in Figure 7, which might be linked to the timeline metaphor 100A in Figure 
5 1A. 

In addition to grouping pictures according to specific events, the 
GUI of the present invention can also provide linked views which group database 
pictures according to stored characteristics such as the location of capture, and 
identifiable persons in pictures. The display 800 in Figure 8 is an example of a 
10 line 802 with proportionately sized location icons 804 linked to a particular year 
(2000 in the example) on the timeline 102 A in Figure 1 A. It is apparent from 
Figure 8 that the many more pictures were captured at "Location 5" than at "Lo- 
cation 4." 

The display 900 in Figure 9 is an example of a line 902 with pro- 

15 portionately sized "people" icons 904 linked to a particular year (2000 in the ex- 
ample) on the timeline 102 A in Figure 1 A. It is apparent from Figure 9 that the 
many more pictures were captured of "Person 5" in that year than of "Person 4." 
For example, "Person 5" might be a newborn baby, while "Person 4" might be a 
person who often takes pictures, but seldom has pictures taken of himself/herself. 

20 When there are no more timeline metaphors, or other picture 

grouping metaphors after a given level in the present-inventive GUI has been 
reached, or when desired by the user (if the viewing screen does not become 
undesirably cluttered), the next level is a display 1000 in Figure 10 showing 
thumbnails representing all of the pictures currently grouped. For example, if the 

25 user activates the "Person 2" icon in the display 900 in Figure 9, thumbnails of all 
of the pictures captured during the particular year (2000 year in the example), and 
featuring or including "Person 2" are displayed. 

Given the disclosure of this Letters Patent, those skilled in the art 
will be able to implement a timeline-based GUI for more efficient and user- 

30 friendly browsing and retrieval of pictures stored in an image database. However, 
a general system 1 100 for implementing the present invention is illustrated in Fig- 
ure 11. The various components of the system 1 100 need not have physical 
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proximity. Indeed, the system 1 100 can be self-contained in a stand-alone com- 
puter system, part of a Local Area Network (LAN), or part of a remote processing 
system using Wide Area Networks (WANs) such as the Internet, to name a few 
variations. 



operation, a large image file memory 1 102 for storing the digital pictures of a 
picture database, along with metadata for each picture, and a metadata decoder 
1 104 for decoding metadata from the image files for use by the system 1 100. A 
GUI generator 1 108 generates the display metaphors of the present invention for 
10 display via a display 1110 under the direction of a user input 1112 and the central 
control 1106. 

Variations and modifications of the present invention are possible, 
given the above description. However, all variations and modifications which are 
15 obvious to those skilled in the art to which the present invention pertains are con- 
sidered to be within the scope of the protection granted by this Letters Patent. 



5 



The system 1 100 includes central control 1 106 to control its overall 
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PARTS LIST 



100A 


Timeline metaphor 


102 A 


Timeline (years) 


104 A 


Year-picture icons 


100B 


Timeline metaphor 


102B 


Timeline (decades) 


106B 


Decade-picture icons 


200A 


Timeline metaphor 


202A 


Timeline (year, months) 


204A 


Month-picture icons 


200B 


Timeline metaphor 


206B 


Timeline (decade, years) 


208B 


Year-picture icons 


300 


Timeline metaphor 


302 


Timeline (month, days) 


304 


Day-picture icons 


400 


Timeline metaphor 


402 


Timeline (day, time) 


404 


Time-picture icons 


500 


Timeline metaphor 


502 


Timeline (day, time block) 


504 


Time block-picture icons 


600 


Timeline metaphor 


602 


Timeline (month, events) 


604 


Event-picture icons 


700 


Timeline (vertical) metaphor 


702 


Timeline fvear events^ 


704 


Event-picture icons 


800 


Timeline metaphor 


802 


Timeline (year, locations) 


804 


Location-picture icons 


900 


Timeline metaphor 
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902 Timeline (year, people) 

904 People-picture icons 

1 000 Thumbnail display 

1 100 Timeline-based Graphical User Interface (GUI) system 

1 1 02 Image file memory 

1 1 04 Metadata decoder 

1 106 Central control 

1108 GUI generator 

1110 Display 

1112 User input 



